Web Survey Bibliography
Despite the best efforts of questionnaire designers, survey respondents don’t always interpret questions as the question writers intended. Researchers have used Grice’s conversational maxims to explain some of these discrepancies. This dissertation extends this work by reviewing studies on the use of Grice’s maxims by survey respondents and describing six new experiments that looked for direct evidence that respondents apply Grice’s maxims. The strongest evidence for respondents’ use of the maxims came from an experiment that varied the numerical labels on a rating scale; the mean shift in responses to the right side of the rating scale induced by negative numerical labels was robust across items and fonts. Process measures indicated that respondents applied the maxim of relation in interpreting the questions. Other evidence supported use of the maxim of quantity — as predicted, correlations between two highly similar items were lower when they were asked together. Reversing the wording of one of the items didn’t prevent respondents from applying the maxim of quantity. Evidence was weaker for the application of Grice’s maxim of manner; respondents still seemed to use definitions (as was apparent from the reduced variation in their answers), even though the definitions were designed to be uninformative. That direct questions without filters induced significantly more responses on the upper end of the scale — presumably because of the presuppositions direct questions carried — supported respondents’ application of the maxim of quality. There was little support for respondents’ use of the maxim of relation from an experiment on the physical layout of survey questions; the three different layouts didn’t influence how respondents perceived the relation among items. These results provided some evidence that both survey “satisficers” and survey “optimizers” may draw automatic inferences based on Gricean maxims, but that only “optimizers” will carry out the more controlled processes requiring extra effort. Practical implications for survey practice include the need for continued attention to secondary features of survey questions in addition to traditional questionnaire development issues. Additional experiments that incorporate other techniques such as eye tracking or cognitive interviews may help to uncover other subtle mechanisms affecting survey responses.
Web survey bibliography - Yan, T. (13)
- The Impact of Response Scale Direction on Survey Responses in a Mixed-mode Survey ; 2016; Hu, M.; Yan, T.; Keusch, F.
- The Impact of Scale Direction, Alignment and Length on Responses to Rating Scale Questions in a Web...; 2016; Keusch, F.; Liu, M.; Yan, T.
- Impact of response scale direction on survey responses in web and mobile web surveys; 2015; Yan, T.; Keusch, F.
- The Effects of the Direction of Rating Scales on Survey Responses in a Telephone Survey; 2015; Keusch, F., Yan, T.
- Analyzing Paradata to Investigate Measurement Error; 2013; Yan, T., Olson, K.
- Should I Stay or Should I go: The Effects of Progress Feedback, Promised Task Duration, and Length of...; 2011; Yan, T., Conrad, F. G., Tourangeau, R., Couper, M. P.
- Nonresponse Error, Measurement Error, And Mode Of Data Collection: Tradeoffs in a Multi-mode Survey...; 2011; Sakshaug, J. W., Yan, T., Tourangeau, R.
- Experiments Exploring the Relationship (or Lack Thereof) Between Nonresponse Error and Measurement Error...; 2008; Tourangeau, R., Groves, R. M., Presser, S., Toppe, C., Kennedy, K., Yan, T.,
- Fast times and easy questions: the effects of age, experience and question complexity on web survey...; 2008; Yan, T., Tourangeau, R.
- Sensitive questions in surveys; 2007; Tourangeau, R., Yan, T.
- How successful I am depends on what number I get: The effects of numerical scale labels and need for...; 2006; Yan, T.
- Gricean effects in self-administered survey. Ph.D. Dissertation; 2005; Yan, T.
- An Experimental Comparison Of Web And Telephone Surveys; 2005; Fricker, S., Galesic, M., Tourangeau, R., Yan, T.